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Abstract. The goal of this study was to compare how weblogs and traditional hand¬ 
written reflective learning protocols compare regarding the use of cognitive and 
metacognitive strategies for knowledge acquisition as well as learning gains in 
secondary school students. The study used a quasi-experimental control group design 
with repeated measurements comparing weblogs and text-based reflective journals both 
with and without prompts. During a learning unit on the subject of climate change, 
students were assigned one to four experimental groups with different writing 
assignments and one control group that did not keep a learning protocol of any kind. 
Comparisons of pretest and posttest scores indicate that students in the experimental 
groups collectively outperform the students in the control group. Looking closer 
however, only the groups writing with the guidance of prompts showed better learning 
gains, while groups writing without prompts did not show significant differences when 
compared to the control group. There were no differences with respect to learning gains 
between groups writing weblogs and those writing with paper and pencil when 
supported by prompts. Without prompts however, students in the paper-and-pencil 
writing condition performed better than students writing blogs. For blogging students, 
prompts seemed to be more important to achieve greater learning gains. In addition, 
students showed greater use of cognitive and metacognitive strategies when guided by 
prompts. Also here, there were no differences with regard to the medium of writing. 
Both cognitive and metacognitive strategies were predictors of learning gains. In 
conclusion, the use of prompts can be considered as important scaffold when writing 
weblogs or paper based learning protocols. 
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Introduction 

Weblogs are regarded as a relatively novel and highly promising medium for promoting 
reflective and self-directed learning in higher education (Robertson, 2011; Sim & Hew, 2010) 
as well as in schools (Downes, 2004; Sawmiller, 2010). Weblogs are chronologically 
structured websites that facilitate the creation of individual or collective online journals. 
Additionally, weblogs have been considered to be an important part in the participative 
Web 2.0, where internet users are being facilitated to become authors (O'Reilly, 2005), and 
potential elements of personal learning environments (Attwell, 2007; Dabbagh & Kitsantas, 
2012). In science education, the focus of digital media is currently shifting from digital 
resources to cognitive tools (Jonassen, 1995; Songer, 2010). Rather than delivering and 
displaying information, cognitive tools support student inquiry and meaning making. As 
cognitive tools, weblogs have the potential to support scientific reasoning through written 
reflection and online discussion in a community of inquiry (Angelaina & Jimoyiannis, 2012). 
However, their use for educational purposes raises multiple questions that need empirical 
clarification. First, it is important to determine the potentials that reflective writing has to 
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influence the learning process. Second, the question arises as to whether writing a weblog 
offers any advantages over traditional learning journals. Third, we need to better 
understand how to best provide teacher support for reflective writing with and without a 
weblog. 

Theoretical Framework 

Journal writing and blogging for learning 

In educational settings, writing is a widely used method for improving learning and for 
reflecting on the learning process (Emig, 1977; Galbraith, 1999; Harris et al., 2011). Writing 
allows one to formulate thoughts in an explicit way and, by slowing down the process of 
thinking, to process them more thoroughly. Terms such as "learning diary," "learning 
journal," "learning protocol" and "reflective journal," are used to designate the written 
products when reflective writing is accompanying the learning process on a regular basis 
(Boud, 2001; Hiemstra, 2001; Moon, 2006; O'Connell & Dyment, 2011; Thorpe, 2004). These 
approaches have cognitive as well as metacognitive potentials. 

The cognitive potentials of writing are fostered, when students are expected to write about 
taught material in their own words. In research about writing, this process is generally 
referred to as "writing-to-learn" (Klein, 1999; Newell, 2006) or "epistemic writing" (Bereiter, 
1980) and it has been considered as a promising approach in many school subjects including 
science education (Baker et al., 2008; Rivard, 1994). Writing-to-learn represents an elaborated 
form of text production in which knowledge must be explicitly remembered, selected, 
summarized, structured, connected, put together with previous knowledge, elaborated, and, 
at times, further developed. Metacognitive potentials are promoted, when learning journals 
are used for the documentation of learning processes, the emphasis is on reflection about 
learning pathways and learning strategies. Both aspects have a close connection. Glogger et 
al. (2012) show that the quality and quantity of cognitive strategies that are ascertained from 
learning protocols are good predictors for learning outcomes. 

In the learning process, learning strategies direct the interaction with and processing of 
information, and thus play an important role in learning (Pressley et al., 1985; Pressley & 
Harris, 2006; Weinstein, Acee & Jung, 2011; Weinstein & Mayer, 1986). Learning strategies 
consist of cognitive schemata that are deliberately applied by the learning subject in a 
controlled and purposeful way in order to achieve learning goals. In distinction to learning 
habits or learning styles, strategies suggest a greater degree of deliberateness. Learning 
strategies include all of the student's capacities for being consciously aware of and directing 
their own learning process, and, in this way, of achieving better learning success. Weinstein 
& Mayer (1986) classify learning strategies into five areas: repetition strategies, elaboration 
strategies, organizing strategies, monitoring strategies, and self-regulation strategies. 
Organizing, elaboration, and repetition strategies belong to the so-called cognitive learning 
strategies. They are useful for the direct absorption of information or to improve the 
processing and storage of knowledge. Monitoring and self-regulation strategies play a 
critical role in planning, monitoring, and evaluating one's own learning processes, and thus 
belong to what are known as metacognitive strategies. 

Research on the topic of metacognition differentiates in a similar manner between cognitive 
knowledge and one's knowledge of how to regulate cognitions (Llavell, 1979; Schraw, 1998; 
Veenman, 2011). Metacognition and learning strategies are important prerequisites for self- 
directed learning (Boekaerts, 1999; Borkowski, 1996; Pintrich, 2000; Schunk, 2008; 
Zimmermann & Schunk, 2001), along with the abilities of co-regulation and shared 
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regulation when learning in a social context (Winne, 2011). Learning journals have been 
shown to be a promising method for activating cognitive and metacognitive learning 
activities, both in the university context (Cazan, 2012; McCrindle & Christensen, 1995), and 
in public schools (Harris et al., 2011; Mason, 1998). Learning journals may be hand-written 
or written using digital media, and the journal writing process may be undertaken with or 
without teacher support. 

Compared to hand-written reflective journals, weblogs display a number of differences and 
specific potentials (Downes, 2004; Robertson, 2011; Sawmiller, 2010; Sim & Hew, 2010). 
Digital text entry is less linear, as text elements can subsequently be modified, expanded 
upon, erased, or moved. Additional advantages include the creative possibilities of working 
with formatting; incorporating tables, images, hypertext and multimedia. Studies show, that 
this has a positive impact on the quality of the texts (Goldberg, Russell & Cook 2003; 
Schwarz, 2004). As weblogs are online, they have a number of additional potentials (Andres 
Martinez, 2012). It is possible for them to be managed as desired from any Internet-capable 
end terminal, such as smartphones, thus supporting the possibility of mobile learning. 
Internet-based learning journals are accessible from any location with a potentially global 
reach. Teachers, parents, and learning partners no longer have to gather the journals in hard¬ 
copy form to be able to read them. Online journals can make it possible not only for the 
individual student but also for learning groups to keep a learning journal. Comment and 
trackback functions can be used to promote feedback and establish a community of 
discourse (Xie, Ke & Sharma, 2008). 

Options for instructional support 

Although journaling is a practice-proven learning method, empirical studies make it clear 
that students do not spontaneously or automatically apply essential cognitive and 
metacognitive strategies for self-directed learning (Luckin et al., 2009; Niickles et al., 2004; 
Schwonke et al., 2006). Learning protocols that are written completely freely tend to read 
like content summaries and contain few indications of reflection upon or implementation of 
learning strategies. Research indicates that prompts and scaffolds are necessary to generate 
productive learning activity both in general instructional situations (Askell-Williams, 
Lawson, & Skrzypiec, 2012; King, 1992; Peters & Kitsantas, 2010; Pressley et al., 1992; 
Sitzmann & Ely, 2010) as well as in writing-to-learn interventions (Glogger et al., 2009; Klein 
& Yu, 2013; Niickles, Hubner & Renkl, 2009; Van den Boom, Paas, Van Merrienboer, & Van 
Gog, 2004). In their meta-analysis, Bangert-Drowns et al. (2004) showed that entirely free 
and undirected writing led to only small learning effects. Indeed, writing has a particularly 
positive effect on learning when the writing task encourages the conscious regulation of the 
learning process. Bereiter, Burtis & Scardamalia (1988) show that prompts are important to 
support school children in the development of knowledge-transforming writing. These 
scaffolds can be subdivided into cognitive prompts and metacognitive prompts. Cognitive 
prompts are directed toward a better understanding of learning content. Metacognitive 
prompts encourage conscious reflection about the learning strategies that have been used. 
Prompts may thus be regarded as strategy activators (Reigeluth & Stein, 1983) or as 
scaffolding that helps to orient students in applying learning strategies (Collins et al., 1989). 
However, Hubner, Niickles & Renkl (2007) only recommend the short-term use of such 
prompts. Over the longer term, prompts not only become superfluous for the students, but, 
at times, might even be experienced as thwarting autonomy. As in general theories on 
"human tutoring", the element of "fading" seems to be of some importance (Chi et al., 2001). 
In addition, Thillmann et al. (2009) showed that prompts might be more effective when 
applied at the appropriate time during learning. 
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Hypotheses 

Based on our review of the literature, the aim of our study was to compare hand-written 
learning protocols to electronic learning protocols in the form of weblogs, using different 
levels of instructional support with respect to learning gains and the use of strategies. Based 
on the theoretical reasoning, the following hypotheses were stated: 

HI: Students who write some kind of learning protocol (weblog or paper-based, with or 
without prompts) will achieve greater learning gains than those who do not write learning 
protocols at all. 

H2: Students who write some kind of learning protocol (either weblog or paper-based) with 
the support of prompts will a) achieve greater learning gains and b) show a more intensive 
use of learning strategies than those who write some kind of learning protocol without the 
support of prompts. 

H3: Students who write digital learning protocols with weblogs (either with or without 
prompts) will a) achieve greater learning gains and b) show more intensive use of learning 
strategies than those who write hand-written learning protocols. 

H4: Knowledge related posttest results will be predicted by pretest results, the use of 
learning strategies and the experimental conditions. 

Method 

Sampling 

Five separate 8th grade classes, totaling 104 students from three public middle schools 
(ISCED Level 2, extended level) in the Canton of Zurich, participated in the study. The 
sample was comprised of 58 girls and 46 boys with an age range between thirteen and 
sixteen (M=14.95; SD= .53). In selecting the classes, we took care that the classroom teachers 
had similar professional experience and that, up to the time of the study, no learning 
protocols or similar tools had been used in classroom instruction. In addition, the topic of 
"climate change," which was selected as the subject for the instructional unit because of its 
high degree of complexity, had not previously been the focus of instruction. 

Study Design 

The study is based on a quasi-experimental experimental design with repeated 
measurements (pretest-posttest-design). The 104 students were randomly assigned by class 
to four experimental groups and one control group (Table 1). 


Table 1. Overview of the experimental groups and their sample sizes (n=104) 


Student groups 

Intervention 

n 

Experimental group 1 

Learning protocol on paper with prompts (paper-prompt 
group) 

19 

Experimental group 2 

Learning protocol on weblog with prompts (weblog- 
prompt group) 

24 

Experimental group 3 

Learning protocol on paper without prompts (paper-no 
prompt group) 

22 

Experimental group 4 

Learning protocol on weblog without prompts (weblog- 
no prompt group) 

19 

Control group 

No learning protocol / no prompts 

20 
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Figure 1. Screenshot of the teacher interface to generate prompts for student weblogs (in German) 



Figure 2. Screenshot of the student weblog writing interface (in German) 

The assignment of classes to the study or control group was randomized. A complete 
randomization at the individual level was not possible within the organizational limits of 
this field experiment. 

The students who wrote weblogs worked with Learninglog (www.learninglog.org), a 
software developed specifically for this purpose (Petko, 2011). Learninglog is based upon 
the open-source weblog program Wordpress (www.wordpress.org). A plug-in enhances the 
usual Wordpress program with many specific functions like group administration, writing 
assignments (Figure 1) and password protected entries (Figure 2). With these functions, the 
plug-in provides specific support for implementing the study design as outlined below. 

Test instruments 

For addressing the research questions and hypotheses, we measured a series of variables 
using different procedures. 
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Students’ prior knowledge test 

The students' prior content-related knowledge and their learning gains were assessed by 
means of a knowledge test, which consisted of four question-complexes with an open 
response format. 

1. Can you explain what "climate change" means? 

2. What do we mean by the "greenhouse effect?" What are its causes? 

3. What are the consequences of global climate change on nature and on people's lives? 
Also think about which of these could affect you personally. 

4. In your opinion, what must mankind do to stop global climate change? What could 
you do as an individual to specifically counter it? 

To evaluate the knowledge test we anonymized all of the datasets from the pre- and 
posttests using codes. The evaluation was performed using the SOLO Taxonomy (Biggs & 
Collis, 1982) by two independent raters, who evaluated the open responses using a 6-step 
rating scale regarding the level of correctness, content about key aspects, connection points, 
conclusions, internal consistency, and structuring (0 = no response, 1 = prestructural, 2 = 
unistructural, 3 = multistructural, 4 = relational, 5 = extended abstract). The average kappa 
coefficient for interrater agreement was k = .65, which indicates satisfactory inter-rater 
reliability (Landis & Koch, 1977). The four individual ratings for each test (SOLO 1-4) were 
summated to obtain an overall value (maximum value of the summated index = 20). 

Learning protocols coding scheme 

To evaluate the learning protocols with respect to the use of cognitive and metacognitive 
strategies, we developed a category system derived from an existing instrument developed 
by Hlibner, Niickles and Renkl (2007) for assessing cognitive and metacognitive activity 
across six dimensions. For each learning protocol, the analysis was conducted along a 5-step 
rating scale (1 = not at all, 2 = hardly ever, 3 = somewhat, 4 = clear, 5 = outstanding) in the 
following dimensions: Organization 1: Expressing key points and their connections; 
Organization 2: Establishing relationships between new knowledge elements; Organization 
3: learning protocols are meaningfully structured; Elaboration: Establishing relationships 
and points of connection to previous knowledge; Self-control: monitoring one's own process 
of understanding; Self-regulation: actively dealing with problems of understanding, 
searching for solutions, taking responsibility for one's own learning process. In line with the 
research outlined above, « Organization l-3» and «Elaboration » can be labelled « cognitive 
learning strategies » for they are more closely related to understanding the content at hand, 
while « Self-Control » and « Self-Regulation » are considered to be a rather metacognitive 
strategies as these are seen as more general ways of reflecting on one's learning that goes 
beyond the comprehension of the given topic. The learning protocols were evaluated by two 
raters. The average kappa coefficient was k = .78, which once again indicates substantial 
interrater reliability (Landis & Koch, 1977). 

Conducting the experiment 

The experiment drew on eight 45-minute classroom lessons from a teaching unit called 
"Man and Environment." In the fourth and sixth lessons, the students of the experimental 
groups each spent 45 minutes writing a learning protocol, in accordance with their 
experimental group assignment. The prompt-groups were given seven "prompts" in 
advance for writing their learning protocols, which were intended to stimulate the use of 
both cognitive and metacognitive strategies: 
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1. What key points in today's lesson did you understand? What key points haven't you 
understood yet? Reasons? 

2. Take another brief look through the papers from today's lesson. Formulate the most 
important points in your own words. 

3. Try to organize the materials in your learning protocol in such a way that you can 
readily notice how everything is connected (build common threads / use heading 
and subheadings, etc.). 

4. Before today have you heard, read about, or had some experience with the subject 
that was just taught in class? 

5. What is your opinion about the subject and what other questions would be of interest 
to you? 

6. What parts of today's lesson do you need to think about one more time in order to be 
sure you really understand everything? 

7. What steps could you take for solving any remaining problems in understanding? 

These prompts were adopted from the study by Hiibner et al. (2007) and the language was 
adapted to the age of the target group. The experimental no-prompt groups were given only 
general instructions along the lines of "write a learning protocol about the lesson just 
completed." In the experimental group 1 (paper-and-pencil with prompts) we handed out 
worksheets with these prompting questions to structure students' writing. In the 
experimental group 2 (weblog with prompts) these seven questions were added to the text 
entry window per default each time students composed a new weblog entry. 

Methods of analysis 

All statistical computations were performed using IBM SPSS Statistics 20. The statistical 
analyses were based upon a significance level of alpha < .05. The assessment of effect size 
(partial q2) was done using the methods of described by Cohen (1992; 1988), by which, for 
simple comparisons, .20 < d <. 50 is considered to be a small effect, .50 < d < .80 a medium 
and d > .80 a large effect. For multiple comparisons we use q2 <.06 counting as a small 
effect, .06 < q2 < .13 as a moderate effect, and q2 >.13 will considered to be a large effect. For 
correlation analysis, effect sizes of the r-family are used with r > .10 as a small, r > .30 as a 
medium and r > .50 as a large correlation. For multiple regression analysis, we use f 2 with 
.02 < f 2 <.15 as small, .15 < f 2 <.35 as medium and f 2 >.35 as a large effect. 

Results and Discussion 

Because assignment to the experimental groups was randomized by class and not 
individually, it is important to first compare the groups according to various relevant 
variables, such as age, most recent report card grades (in the subject areas of mathematics, 
language, geography, and biology), intelligence (IQ measured by CFT 20-R: Weiss, 2008), 
and previous knowledge of the specific content in the pretest. No significant differences 
were shown between the groups for any of the variables tested. 


Effects of experimental conditions on content-related learning gains 

Both the experimental groups and the control group showed a significant improvement in 
their knowledge about climate change (Table 2). The effect sizes (Cohen's d) were greater 
than d>2 for the experimental groups and just under this threshold for the control group. 
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Table 2. Means and standard deviations of student's pretest and posttests results on the subject of 
climate change according to the SOLO Index (scoring scale: 0 - 20) by groups 


Source 

Pretest Content-specific 
knowledge (SOLO-Index) 
tl 

Posttest Content-specific- 
knowledge (SOLO-Index) 
t2 

M 

SD 

M 

SD 

(1) Paper-prompt group 

7.42 

(1.54) 

12.53 

(1.81) 

(2) Weblog-prompt group 

7.42 

(1.82) 

12.00 

(2.11) 

(3) Paper-no-prompt group 

7.73 

(1.39) 

11.41 

(1.68) 

(4) Weblog-no-prompt group 

6.37 

(2.09) 

10.68 

(1.97) 

(5) Control group 

7.30 

(1.75) 

10.40 

(1.76) 


To test the differences, we used a repeated measures ANOVA, with group membership as 
the between-subjects factor and the test results at tl and t2 as the within-subjects factor. The 
overall interaction effect between "learning protocol conditions" and "test results" turns out 
to be significant, F(4, 99) = 4.23, p < .01, partial r|2 = .15. In detail, the post-hoc test (LSD) 
shows a number of significant group differences with respect to their learning gains (p < 
.05). Students in the "paper with prompts" group achieved significantly greater learning 
gains than the students in the "weblog without prompts" group or the students in the 
control group. The "paper without prompts" intervention group achieved significantly 
greater learning gains than the "weblog without prompts" intervention group. Moreover, 
there was a significant difference between the means in the "weblog with prompts" group 
and the "weblog without prompts" group in favor of the intervention group with prompts 
(p < .05). Based on these findings, we are inclined to conclude that HI has been disproved as 
it turned out that not all experimental groups achieved better than the control group. 
However, if we compare learning gains of the treatment groups taken together (N=84, 
Mtl=7.26, SDtl=1.76, Mt2=11.67, SDt2=1.98) with the scores of the control group (N=20, 
Mtl=7.30, SDtl=1.75 ; Mt2= 10.40, SDt2 =1.76) we find a significant advantage for the 
experimental groups who write learning protocols, F(l, 102)=9.05, p<.01, partial r|2 = .08. 
However this supports HI, this observation could be an effect of the prompting rather than 
the learning protocol writing. 

To test the hypotheses H2a and H3a, a priori contrasts have been computed (Tabachnick & 
Fidell, 2012). Supporting hypothesis H2a, the prompt groups achieved significantly greater 
learning gains than the no-prompt groups, F(l, 99) = 5.19, p < .05, partial r|2 = .05, and the 
control group, F(l, 99) = 5.28, p < .05, partial r|2 = .05. The no-prompt groups did not show 
greater knowledge gains than the control group, F(l, 99) = .44, ns.. To check hypothesis H3a, 
we tested whether the potential differences in learning gains might be associated with the 
writing medium. The subsequent analysis of variance showed no significant difference 
between the weblog-prompt group and the paper-prompt group with respect to learning 
gains, F(l, 99) = .27, ns. Finally, the paper-no-prompt group achieved significantly greater 
learning gains than the weblog-no-prompt group, F(l, 99) = 4.38, p <.05, partial q2 = .04. 

Effects of the experimental conditions on the use of cognitive and metacognitive 
strategies in the learning protocols 

In the following chapter, we present the results for the statistical analysis on the question 
whether the choice of the writing medium and the support of prompts have an effect on the 
use of learning strategies in the first learning protocol (LP1) and the second learning 
protocol (LP2) as rated by the researchers. The descriptive analysis shows that there are 
some differences in the means between the experimental groups (Table 3). 



Supporting learning with weblogs in science education 11 


Table 3. Student's display of learning strategies in their writing (rating scale: 1 - 5): Means and 
standard deviations for the first learning protocol (LP1) and the second learning protocol (LP2) by 

groups 


Variables 

Paper 

prompt 

group 

Weblog 

prompt 

group 

Weblog 

no-prompt 

group 

Paper 

no-prompt 

group 

M (SD) 

M (SD) 

M (SD) 

M (SD) 

Organizational strategies 1 (key points) in LP1 

3.79 (.79) 

3.46 (1.02) 

3.47 (.51) 

3.27 (.88) 

Organizat. strategies 2 (internal structure) in LP1 

3.68 (.89) 

3.17 (.70) 

3.16 (.69) 

3.86 (.83) 

Organizat. strategies 3 (external structure) in LP1 

3.05 (.85) 

2.88 (.99) 

3.63(1.01) 

2.27(1.20) 

Elaboration strategies in LP1 

2.84 (1.21) 

2.58 (1.02) 

1.16 (.38) 

1.50(1.01) 

Self-control strategies in LP1 

2.95 (1.39) 

2.83 (1.13) 

1.05 (.23) 

1.64 (.95) 

Self-regulation strategies in LP1 

2.58 (1.39) 

2.12 (1.15) 

1.00 (.00) 

1.05 (.21) 

Organizational strategies 1 (key points) in LP2 

3.58 (.90) 

3.37 (.92) 

3.16 (.50) 

3.59 (.96) 

Organization strategies 2 (internal structure) LP2 

3.53 (.61) 

3.25 (.94) 

2.95 (.78) 

3.82 (.73) 

Organization strategies 3 (external structure) LP2 

2.89 (.81) 

3.08 (1.06) 

3.05(1.13) 

2.00(1.11) 

Elaboration strategies in LP2 

2.75 (1.33) 

2.37 (1.12) 

1.11 (.32) 

1.18 (.66) 

Self-control strategies in LP2 

2.53 (1.22) 

2.58 (1.02) 

1.00 (.00) 

1.68(1.09) 

Self-regulation strategies in LP2 

1.58 (.90) 

2.00 (.98) 

1.00 (.00) 

1.00 (.00) 


When testing hypothesis H2b, we found significant differences between the prompt groups 
and the no-prompt groups with respect to the number of cognitive and metacognitive 
learning strategies used in their learning protocols. This is the case for both Learning 
protocol 1, Pillai's Trace = .51, F(7, 77) = 13.34, p < .001, and Learning protocol 2, Pillai's 
Trace = .45, F(6, 77) = 10.46, p < .001. Follow-up analyses using Mann-Whitney's U test 
showed that for both learning protocols, the prompt-groups differed to a highly significant 
degree from the no-prompt-groups in the strategy dimensions "Elaboration", "Self¬ 
regulation", "Self-control" and "Organizational strategies 3" for LP2. For the use of 
organizational strategies, no significant differences were demonstrated between the two 
groups. 

To test hypothesis H3b, we examined whether the weblog-prompt group showed a greater 
number of cognitive and metacognitive strategies than the paper-prompt group. The 
multivariate analysis showed that the weblog-prompt group and the paper-prompt group 
did not differ significantly from each other with respect to the number of cognitive and 
metacognitive learning strategies in the learning protocols. This was the case for both 
Learning protocol 1, Pillai's Trace = .90, F(6,36) = .59, ns., and for Learning protocol 2, Pillai's 
Trace = .10, F(6,36) = .66, ns. Because of the non-significant findings, we did not perform a 
follow-up analysis. Additionally, we tested whether the learning protocols of the weblog-no 
prompt group contained a greater number of cognitive and metacognitive strategies than 
those of the paper-no-prompt group. Our multivariate analysis of variance showed that the 
weblog-no-prompt group and the paper-no-prompt group differed significantly from each 
other with respect to the number of cognitive and metacognitive learning strategies used. 
This effect was seen in both the first learning protocol, Pillai's Trace = 44, F(6,34) = 4.40, p < 
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.005, and in the second learning protocol, Pillai's Trace = .60, F(6,34) = 8.46, p < .001. Follow¬ 
up analysis using Mann-Whitney's U test showed that the groups differed significantly in 
Learning Protocols 1 and 2 with respect to the variables "Organizational Strategy 2", for 
LP1, U = 113.50, p < .01; Organizational Strategy 2, for LP 2, U = 93.50, p < .01, 
"Organizational Strategy 3," for LP1, U = 92.00, p < .01; "Organizational Strategy 3," for LP2: 
U = 109.00, p < .05, and also for "Self-control" for LP1, U = 141.50, p < .05; "Self-control," for 
LP 2: U = 142.500, p < .01. However, for learning protocols without prompts, only the effect 
for the organizational strategy "meaningfully structuring learning protocols" turned out to 
favor learning protocols written online. 

To test H4, we computed correlations between the observed learning strategies and the 
knowledge scores in tl, t2 as well as the knowledge gain score. For this analysis, we 
computed a mean score for each type of learning strategy across the two learning protocols. 
To test our hypothesis 4, Pearson correlations with a Bonferroni-corrected two-tailed 
significance level of p < .05 were employed (Table 4). 

Table 4 shows moderately positive and significant correlations between cognitive learning 
strategies (1) and (2) and posttest knowledge scores. The strategy (1) of summarizing key 
points was also positively correlated with knowledge gains. Metacognitive learning 
strategies did show moderate but not quite significant correlations with posttest scores and 
knowledge gains. Learning strategies did not seem to be related to pretest scores. 

To explore these findings further, we computed a linear regression with posttest scores as 
the dependent variable and all others as independent variables. Also, we included 
experimental group membership as dummy coded additional variables. The final model 
with significant predictors only is shown in Table 5. The regression analysis showed that 
51% of the variance in posttest knowledge scores can be explained with three variables: 
pretest content specific knowledge, the application of learning strategies focusing on the 
writing of key points and self-control strategies, i.e. monitoring one's own understanding 
and comprehension difficulties. 

With an f 2 = 1.04 this can be considered to be a large effect. Other learning strategies turned 
out as not significant, mostly due to issues of collinearity. Also, the inclusion of the dummy 
coded experimental group membership variables did not improve the model. 


Table 4. Pearson correlations between learning strategies observed in the learning protocols and 
weblogs, pre- and posttest-knowledge and knowledge gain scores 



(1) 

(2) 

(3) 

(4) 

(5) 

(6) (7) (8) 

(1) Organisational strategies 1 (key points) 







(2) Organisational strategies 2 (internal structure) 

62 *** 






(3) Organisational strategies 3 (external structure) 

.24 

.01 





(4) Elaboration strategies 

.04 

-.04 

.07 




(5) Self-control strategies 

.07 

.11 

.03 

.68*** 



(6) Self-regulation strategies 

-.12 

-.09 

.06 

.63*** 



(7) Prestest Content-specific knowledge (SOLO tl) 

.05 

.17 

-.14 

.21 

.08 

.13 

(8) Posttest Content-specific-knowledge (SOLO t2) 

.35* 

.36* 

-.01 

.32 

.34 

.25 .54*** 

(9) Gain Score (SOLO-Index t2-tl) 

.35* 

.24 

.13 

.15 

.31 

.16 -.37** .58*** 


*p < .05, **p < .01, ***p < .001, with Bonferroni corrections 
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Table 5. Linear regression with Posttest Content-specific knowledge (SOLO-Index t2) as 

dependent variable 


Predictors 

R 2 

b 

SE 

beta 

Post-test Content specific knowledge, F(3, 80) = 27.89*** 

(7) Pre-test Content-specific knowledge (SOLO tl) 

.51 

.61 

..09 

54 *** 

(1) Organisational strategies 1 (key points) 


.86 

.22 

54 *** 

(5) Self-control strategies 


.48 

.14 

2g** 


N= 84, *p < .05, **p < .01, ***p < .001 


Conclusions 

The study tried to answer the question whether weblogs are equally if not better suited to be 
used as a learning tool when compared with traditional learning protocols on paper. In 
theory, weblogs are expected to be more efficient due to their advantages as a digital online 
medium for writing and discussion (Downes, 2004; Robertson, 2011; Sawmiller, 2010; Sim & 
Hew, 2010). In combination, we wanted to test the assumption that prompts given by the 
teacher lead to better learning gains and a more intensive application of learning strategies 
in weblogs as well as in traditional learning protocols (Glogger et al., 2009; Klein & Yu, 2013; 
Nuckles, Hiibner & Renkl, 2009; Van den Boom et al., 2004). The findings of our field 
experiment with 104 secondary school students over a period of eight lessons show that 
although, taken together, our experimental groups outperformed the control group, not all 
variants of writing learning protocols compared equally favorable (HI disproved). Instead, 
it seems to matter how the writing process was guided. Students who wrote their learning 
protocols with the support of prompts not only achieved greater learning gains in 
comparison with the control group, but also in comparison with those experimental groups 
that wrote learning protocols without instructional support through prompts (H2a 
confirmed). In the learning protocols written by the prompts groups, there were a greater 
number of contributions that indicated the specific use of metacognitive and cognitive 
strategies (H2b confirmed). 

The present study thus underscores the finding already shown in other studies (e.g. 
Nuckles, Hiibner & Renkl, 2009) that prompts are of great significance for learning and 
learning-related reflection through writing. As a new finding that goes beyond previous 
results, our study showed that the choice of writing medium - weblog versus paper and 
pencil - had no effect on learning gains as long as writing was supported by prompts (H3a 
disproved). However, when writing the learning protocols took place without the support of 
prompts, the experimental group that wrote learning protocols on paper achieved greater 
learning gains than the group that wrote their protocols online. This finding is surprising, 
and it is difficult to reach a plausible explanation without further inquiry. It is possible that 
technical difficulties that arose in using the weblog software were responsible for the fact 
that learners were better able to concentrate on learning content when using paper and 
pencil. When a rather novel medium like weblogs are used, prompts may serve as activators 
to overcome possible negative effects that might come along with the uncertainties of the 
new writing situation. The analysis of the written content shows that a significantly greater 
number of organizational strategies that were aimed at arranging the text were used in the 
electronic learning protocols when compared to their paper and pencil counterparts (H3b 
confirmed in one sub-dimension). This finding is less surprising, because a specific 
advantage of computer based writing versus handwritten text creation is that it makes it 
possible to develop the text in a non-linear fashion and to subsequently rearrange and 
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correct existing texts. As shown in our regression analysis, pretest-knowledge, 
organizational strategies and self-control-strategies accounted for 51% in the variance in 
posttest-knowledge. While no direct effects of treatment group membership were found, 
there still might be mediated effects that need further investigation. 

In summary, the study shows that writing learning protocols leads to greater learning gains 
if the writing is guided with prompts. Prompts also have a positive effect on the application 
of cognitive and metacognitive strategies that lead to better knowledge gains. By contrast, 
unguided writing often results in a general content summary of the instructional materials, 
but not in deeper information processing or reflection about what was learned. The selection 
of the writing medium (weblog or paper) appears to play a subordinate role as compared to 
instructional support by means of prompts. Weblogs can even show worse effects than 
paper-and-pencil writing if not supported by prompts. This should lead to some caution, 
when employing weblogs as a new substitute for traditional learning protocol approaches. 
Future research needs to focus on the specifics of how writing weblogs for learning can be 
supported by teachers. 

Of course, our study has a number of limitations that need to be taken into account. Not all 
features of the weblogs were included in the intervention. For example, the option to 
mutually comment on each other's entries was not employed while this might be a central 
element of a successful blogging experience (Ellison & Wu, 2008; Hall & Davison, 2007). 
Similarly, tagging and the categorization of entries were not part of the present study, nor 
were the options for managing learning protocols not only as text but also in the form of 
drawings, mind maps, audio or video, and for linking them with the option of mobile 
learning. Also, we have to assume that the length of the intervention was too brief for the 
learners to become accustomed to their weblogs or to turn them into an integral part of their 
"personal learning environment". It is possible that novelty effects and operational 
difficulties could have played a role. In addition, prompts were presented as a writing 
instruction in advance. This might also be a limiting factor as it seems to be better to present 
prompts adaptively during the writing intervention (Thillmann et al., 2009). Beyond this, the 
present study also has certain methodological limitations. Since complete randomization at 
the individual level was not possible within the quasi-experimental setting, there is a risk 
that the effects of independent variables could have been confounded by other variables- 
even though we did make a deliberate attempt to control for various independent variables. 
In addition, the sample sizes are relatively small. Yet the fact that they show significant 
differences despite the small intervention groups may be an indicator that research on 
prompts for blogging may be an important area for future research. 
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